Sentence level intelligibility evaluation for Mandarin text-to-speech systems using semantically unpredictable sentences

Intelligibility assessment is one of the important aspects in the text-to-speech system (TTS) evaluation. Several intelligibility assessment methods have been proposed and successfully applied to European languages, both at word level and sentence level. Since Mandarin has its own unique features, these methods must be modified when applying to Mandarin. The word level assessment methods such as DRT and MRT have successfully been modified and extended to Mandarin (e.g. CDRT, CDRT-tone and CMRT). Sentence level assessment methods, on the other hand, have not been well studied for Mandarin. This paper focuses on the Semantically Unpredictable Sentences (SUS) test, which is one of the most commonly used sentence level assessment methods, and considers several important aspects of the SUS test design when extending it to Mandarin. It also compares the SUS test for Mandarin with CDRT, CDRT-tone and CMRT.